ABSTRACT
Introduction: The comparison of safety profiles for products recently on the market is difficult. There is a lack of methodology for quantifying the potential differences between products that have the same indication. Objective(s): Provide the tools to quantify the differences in spontaneous reporting between two products Methods: Under the null assumption that two products have the same safety profile, the scatterplot of the Empirical Bayes Geometric Mean (EBGM) measured for the different MedDRA Preferred Terms (PTs) post Product A (axis x) and post Product B (axis y) should follow the diagonal line. An Euclidian distance from the EBGM to the diagonal line measures the deviation from what would have been expected under the null assumption of similar safety profiles. As the deviation does not capture the statistical uncertainty around the estimate, we propose as measure of the deviation the minimal distance of the four Euclidian distances calculated from each of the credibility intervals around the EBGM post Product A and Product B. Result(s): We quantified the most significant differences in reporting between the two vaccines that were approved in the US against covid- 19 using publicly available data from the Vaccine Adverse Event Reporting System (VAERS). A visualization capturing the global trend of the most substantial differences in reporting was generated. Conclusion(s): This relatively simple method can provide quantification of the differences in reporting and could help prioritize one product over the other for some population subgroups.
ABSTRACT
Introduction: A method of time-to-onset (TTO) signal detection for screening unexpected temporal patterns from vaccine spontaneous report data has been published in 2012 [1]. EMA listed TTO analysis as one of the methods to be considered for signal detection [2]. Due to the large number of spontaneous reports associated to covid-19 vaccines, highly significant TTO signals could be detected whereas there are no clinically relevant unexpected temporal patterns. Objective(s): Objective was to revise the original method so that it is less sensitive to small differences and test it on two covid-19 vaccines (Pfizer/BioNTech and Moderna). Method(s): The revised method used only the most predictive measure [3] of the two Kolmogorov-Smirnov (KS) tests originally designed: the p-value of the KS test of the TTO distribution of a given event post a given vaccine against the TTO distribution of the same event post other vaccines. A threshold on the Kolmogorov-Smirnov distance, that can have values between 0 and 1-0 for no difference between time-to-onset distributions and 1 for extreme differences- was set at 0.2. A threshold on the p-value of the KS test was set at 0.05. The Vaccine Adverse Event Reporting System was prospectively frozen every week of the first quarter 2021 and the revised TTO signal detection method was prospectively applied on the two covid-19 vaccines. The performance in detecting events that were posteriori determined as causally related to the exposure of the covid-19 vaccines, namely Pericarditis and Myocarditis, was retrospectively assessed. Result(s): Pericarditis post Pfizer/BioNTech covid-19 vaccine emerged as a significant TTO signal as early as 15JAN21 (N: 4, distance: 0.80, p-value: 0.012). Myocarditis post Pfizer/BioNTech covid-19 vaccine emerged as a significant TTO signal as early as 29JAN21 (N:5, distance: 0.59, p-value: 0.031). Pericarditis post Moderna covid-19 vaccine emerged as a significant TTO signal as early as 12FEB21 (N: 13, distance: 0.55, p-value: 0.00093). Myocarditis post Moderna covid-19 vaccine emerged as a significant TTO signal as early as 05FEB2021 (N:7, distance: 0.73, p-value: 0.0014). Conclusion(s): The revised TTO method allowed early detection of unexpected TTO patterns post exposure to covid-19 vaccines by controlling both the level of significance and the magnitude of difference between the TTO distributions in a context of mass vaccination where individual case review is challenging.
ABSTRACT
Introduction: A method of time-to-onset (TTO) signal detection for screening unexpected temporal patterns from vaccine spontaneous report data has been published in 2012 [1]. EMA listed TTO analysis as one of the methods to be considered for signal detection [2]. Due to the large number of spontaneous reports associated to covid-19 vaccines, highly significant TTO signals could be detected whereas there are no clinically relevant unexpected temporal patterns. Objective: Objective was to revise the original method so that it is less sensitive to small differences and test it on two covid-19 vaccines (Pfizer/BioNTech and Moderna). Methods: The revised method used only the most predictive measure [3] of the two Kolmogorov-Smirnov (KS) tests originally designed: the p-value of the KS test of the TTO distribution of a given event post a given vaccine against the TTO distribution of the same event post other vaccines. A threshold on the Kolmogorov-Smirnov distance, that can have values between 0 and 1-0 for no difference between time-to-onset distributions and 1 for extreme differences- was set at 0.2. A threshold on the p-value of the KS test was set at 0.05. The Vaccine Adverse Event Reporting System was prospectively frozen every week of the first quarter 2021 and the revised TTO signal detection method was prospectively applied on the two covid-19 vaccines. The performance in detecting events that were posteriori determined as causally related to the exposure of the covid-19 vaccines, namely Pericarditis and Myocarditis, was retrospectively assessed. Results: Pericarditis post Pfizer/BioNTech covid-19 vaccine emerged as a significant TTO signal as early as 15JAN21 (N: 4, distance: 0.80, p-value: 0.012). Myocarditis post Pfizer/BioNTech covid-19 vaccine emerged as a significant TTO signal as early as 29JAN21 (N:5, distance: 0.59, p-value: 0.031). Pericarditis post Moderna covid-19 vaccine emerged as a significant TTO signal as early as 12FEB21 (N: 13, distance: 0.55, p-value: 0.00093). Myocarditis post Moderna covid-19 vaccine emerged as a significant TTO signal as early as 05FEB2021 (N:7, distance: 0.73, p-value: 0.0014). Conclusion: The revised TTO method allowed early detection of unexpected TTO patterns post exposure to covid-19 vaccines by controlling both the level of significance and the magnitude of difference between the TTO distributions in a context of mass vaccination where individual case review is challenging.
ABSTRACT
Introduction: The comparison of safety profiles for products recently on the market is difficult. There is a lack of methodology for quantifying the potential differences between products that have the same indication. Objective: Provide the tools to quantify the differences in spontaneous reporting between two products Methods: Under the null assumption that two products have the same safety profile, the scatterplot of the Empirical Bayes Geometric Mean (EBGM) measured for the different MedDRA Preferred Terms (PTs) post Product A (axis x) and post Product B (axis y) should follow the diagonal line. An Euclidian distance from the EBGM to the diagonal line measures the deviation from what would have been expected under the null assumption of similar safety profiles. As the deviation does not capture the statistical uncertainty around the estimate, we propose as measure of the deviation the minimal distance of the four Euclidian distances calculated from each of the credibility intervals around the EBGM post Product A and Product B. Results: We quantified the most significant differences in reporting between the two vaccines that were approved in the US against covid19 using publicly available data from the Vaccine Adverse Event Reporting System (VAERS). A visualization capturing the global trend of the most substantial differences in reporting was generated. Conclusion: This relatively simple method can provide quantification of the differences in reporting and could help prioritize one product over the other for some population subgroups.